Under the RAM model [1], the "time" an algorithm takes is measured by the elementary operations of the algorithm. While users and developers may concern more about the wall clock time an algorithm takes to train the models, it would be fairer to use the standard worst case computational time complexity to compare the time the models take to train. Using computational complexity has the benefits of ignoring the differences like the computer power and architecture used at runtime and the underlying programming language, allowing users to focus on the fundamental differences of the elementary operations of the algorithms. Note that the time complexity can be very different during training and testing. For example, parametric models like linear regression could have long training time but they are efficient during test time.

How do you compare the estimated accuracy of different machine learning algorithms effectively? In this post you will discover 8 techniques that you can use to compare machine learning algorithms in R. You can use these techniques to choose the most accurate model, and be able to comment on the statistical significance and the absolute amount it beat out other algorithms. Discover how to prepare data, fit machine learning models and evaluate their predictions in R with my new book, including 14 step-by-step tutorials, 3 projects, and full source code. Compare The Performance of Machine Learning Algorithms in R Photo by Matt Reinbold, some rights reserved.

It is important to compare the performance of multiple different machine learning algorithms consistently. In this post you will discover how you can create a test harness to compare multiple different machine learning algorithms in Python with scikit-learn. You can use this test harness as a template on your own machine learning problems and add more and different algorithms to compare. How To Compare Machine Learning Algorithms in Python with scikit-learn Photo by Michael Knight, some rights reserved. How do you choose the best model for your problem?